METHOD AND SYSTEM FOR RECOGNIZING AND SELECTING A REGION 
OF INTEREST IN AN IMAGE 



FIELD OF THE INVENTION 

[0001] The present invention relates generally to a method and system for 
recognizing and selecting at least one region of interest in an image. 

BACKGROUND OF THE INVENTION 

[0002] Computer-generated images find use in many different industries 

to model surfaces and solids. In the medical field, computer imaging is used in 
combination with ultrasound imaging, magnetic resonance imaging or other 
medical imaging technology to display, analyze and organize the data these 
medical imaging technologies provide. For example, ultrasound machines use 
ultrasonic wave, i.e. sonar, to scan a patient's body. The data thus obtained is 
then analyzed by physicians to assist the physicians in their diagnosis and 
treatment of patients. Ultrasound can be used to view a fetus, blood-flow 
patterns in arteries, or to scan organs for irregularities such as cysts, etc. 

[0003] Typically, a three dimensional image is displayed to a user by being 
projected on a two dimensional surface such as a screen or a print out. 
Computer systems have been devised that permit the user to take different 
cross-sectional views of the image, thereby enabling a user to view a portion of 
the three dimensional image by choosing one or more cross-sectional views. 
Some computer systems permit the user to geometrically transform the image 
by rotating, translating and scaling the image, and also permit the user to take 
cross-sectional views at different orientations, such that the user can sequentially 
view the three-dimensional image using a series of oblique planes. Other 
computer systems take a different approach; instead of showing a cross-section 
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of the three dimensional image, they "render" the volume by making part of the 
volume transparent or translucent such that points under those parts are 
revealed. In a maximum intensity display, for example, the highest intensity 
points along a line parallel to the line of sight of the user are shown, similar to an 
x-ray picture of a foot. 

[0004] While three dimensional images provide a wealth of information, 
filtering out the information that is of interest from irrelevant information can be 
very time-consuming. This is especially the case for medical images in which the 
feature of interest is often obscured from clear view by other physical features. 
A typical method of eliminating irrelevant portions of three-dimensional images 
is through user selection of portions of the image to scalpel away from a 
displayed view of the image. Based upon one or more displayed views of the 
image, the user selects a series of points defining an enclosure. In a two- 
dimensional image, the selected points define a shape, whereas in a three- 
dimensional image, the selected points define a volume. The image processing 
system then eliminates any image data within the selected enclosure and re- 
displays the view(s) of the image. The user may then select points to define 
another enclosure to eliminate. In such a manner, the user pares away 
unwanted portions of the image. Among the disadvantages of this method are 
the time required of the user to select the enclosures and the difficulty of 
accurately selecting the unwanted areas without also scalpelling away portions 
of the feature of interest. 

[0005] Accordingly, there remains a need for a system and method of 

quickly and easily analyzing and manipulating computer-generated images in 
order to filter out as much irrelevant information as possible from the images and 
display only the region of interest. 
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BRIEF SUMMARY OF THE INVENTION 



[0006] In one aspect, the present invention provides a method of selecting 

a portion of an image, the portion of the image containing a feature of interest, 
a view of the image being displayed on a display, the image being generated 
from an image-defining array containing image data corresponding to image 
properties at points within a coordinate system containing the image, the image- 
defining array being stored by an image processing system, and a user having 
a user interface means for interaction with the image processing system, the 
method including the steps of determining a seed point in the image defining 
array in response to the selection by the user of a pixel in the view of the image 
being displayed, and growing a region about the seed point, wherein the growth 
of the region is responsive to the user interface, such that manipulation of the 
user interface selectively determines the extent of the growth of the region. 

[0007] In a further aspect, the present invention provides a method of 

selecting a desired portion of an image, as described above, wherein the step of 
growing a region includes an iterative series of steps, including the steps of 
evaluating untested points in the image defining array adjacent to a boundary 
member point to determine if each untested point is a member of the region, the 
boundary member point being the seed point for a first iteration and the boundary 
member point being a member point at the boundary of the region for 
subsequent iterations; adding to the region those untested points that are 
determined to be members of the region; excluding from the region those 
untested points that are determined not to be members of the region; and visually 
distinguishing, on the view of the image being displayed, the points determined 
to be members of the region from other points, and wherein the number of 
iterations performed is responsive to the user interface means, such that 
manipulation of the user interface means selectively determines the extent of the 
growth of the region. 
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[0008] In yet another aspect, the present invention provides an image 
processing system for selecting a portion of an image, the portion of the image 
containing a feature of interest, a view of the image being displayed on a display, 
the image being generated from an image-defining array containing image data 
corresponding to image properties at points within a coordinate system 
containing the image, the image-defining array being stored by the image 
processing system, a seed point in the image defining array being determined 
based upon the selection by a user of a pixel in the view of the image being 
displayed, the image processing system including a user interface for selecting 
the pixel and for selectively determining the extent of the growth of a region, and 
a region growing module for growing the region about the seed point, the region 
□ growing module increasing and decreasing the growth of the region in response 

,"j to manipulation of the user interaction means, wherein the view of the image 

J* being displayed, shows the size of the region. 

[0009] In yet another aspect, the present invention provides a computer 

1,. program product comprising a computer readable medium carrying program 

fy means for selecting a desired portion of an image, the desired portion of the 

[7 image containing a feature of interest, a view of the image being displayed on a 

7 display, the image being generated from an image-defining array containing 

image data corresponding to image properties at points within a coordinate 
system containing the image, the image-defining array being stored by an image 
processing system, and a user having a user interface means for interaction with 
the image processing system, the computer program product including code 
means for determining a seed point in the image defining array in response to the 
selection by the user of a pixel in the view of the image being displayed; code 
means for iteratively growing a region about the seed point, wherein the number 
of iterations performed is responsive to the user interface means, such that 
manipulation of the user interface means selectively determines the extent of the 
growth of the region; and code means for displaying a view of a portion of the 
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image corresponding to the region. 

[0010] Other aspects and features of the present invention will become 

apparent to those ordinarily skilled in the art upon review of the following 
description of specific embodiments of the invention in conjunction with the 
accompanying figures. 



BRIEF DESCRIPTION OF THE DRAWINGS 

[0011] Reference will now be made, by way of example, to the 

accompanying drawings, which show preferred aspects of the present invention, 
and in which: 

[0012] Figure 1 is a block diagram showing an optimal view selection 

system according to an aspect of the present invention; 

[0013] Figure 2 is a screen shot showing features of a graphical interface 

for a system according to an aspect of the present invention; 

[0014] Figure 3 is a screen shot showing features of a graphical interface 
for a system according to an aspect of the present invention; 

[0015] Figure 4 is a screen shot showing features of a graphical interface 

for a system according to an aspect of the present invention; 

[001 6] Figure 5 is a screen shot showing features of a graphical interface 

for a system according to an aspect of the present invention; 

[0017] Figure 6 is a screen shot showing features of a graphical interface 

for a system according to an aspect of the present invention; 

[0018] Figure 7 is a screen shot showing features of a graphical interface 

for a system according to an aspect of the present invention; 
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[0019] Figure 8 is a screen shot showing features of a graphical interface 

for a system according to an aspect of the present invention; 

[0020] Figure 9 is a screen shot showing features of a graphical interface 
for a system according to an aspect of the present invention; and 

[0021 ] Figure 1 0 is a screen shot showing features of a graphical interface 

for a system according to an aspect of the present invention. 

DETAILED DESCRIPTION OF AN EMBODIMENT OF THE INVENTION 

[0022] The present invention comprises a system and method for 
recognizing and selecting a region of interest in an image. In one embodiment, 
the system and method employs a perspective view and selectable sectional 
views to assist the user to recognize a feature of interest and to identify 
unwanted features. The system and method also employs region growing to 
select a region containing the feature of interest and excluding the unwanted 
artifacts. The region growing proceeds by iterative growth steps, the number of 
steps being controlled by the user through a user interface device. 

[0023] Referring to Figure 1 there is illustrated a block diagram of a 

computer system 22 for analyzing computer-generated three dimensional images 
in accordance with a preferred embodiment of the invention. As shown in Figure 
1, the computer 22 is connected to a display 24 (for example, a 640 by 480 
screen), an input device 30 such as a manually operated mouse 30, and an 
ultrasound image data source 32 such as an ultrasound system that acquires 
ultrasound image data. 

[0024] When scanning a patient, an ultrasound operator passes a probe 

over a portion of the patient's body. The probe emits and receives a series of 
ultrasonic waves. Based on the difference between the ultrasonic waves that are 
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emitted and those that are received, a frame of data representing a cross- 
sectiona! view of the patient's body is obtained along the path of the probe. The 
probes generates, say, 150 frames of data in a typical pass over the scanned 
portion of the patient's body. Each frame of data represents a cross-section of 
the part of the patient's body that is scanned using ultrasonic waves. Each frame 
is typically 640 by 480 pixels, but only a portion of the frame corresponding to the 
area of interest, which is typically about 250 by 250 pixels, is stored. Accordingly, 
the image data in the ultrasound image data source 32 for a single image would 
consist of about 150 frames, each frame being about 250 by 250 pixels. 

[0025] The image data source 32 communicates with the computer 22 and 
provides image data to a data submodule 28 of the imaging software (not 
represented in the block diagram of Figure 1) on the computer 22. The data 
submodule 28 orders the image data in an image data array, such that each 
ordered image property in the image data array has associated spatial 
coordinates. The x and y spatial coordinates are determined by the location of 
the data within the 250 by 250 frame, while the z spatial coordinate is assigned 
to the data by the data submodule 28 based on the frame in which the data is 
found. 

[0026] In orderto form three dimensional images based on the ultrasound 

data for image data received from the ultrasound image data source 32, the 
computer 22 includes a conventional polytope submodule 34 for generating a 
coordinate space. Preferably, the coordinate space takes the form of a right- 
angled parallelepiped, which will be referred to as modelPoly (where "poly" is an 
abbreviation of "polyhedron"). The modelPoly is defined around the origin of a 
left-handed xyz coordinate system. 

[0027] The polytope submodule 34 handles all of the geometry involved 

in rotating, translating and scaling modelPoly. The calculations involved in 
geometrically transforming the coordinate space are more easily performed when 
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modelPoly remains located about the origin. Accordingly, all of the geometry 
involved in rotating, translating and scaling modelPoly is recorded in a rotation 
matrix R. a translation matrix T and a scaling matrix S, while modelPoly itself 
remains located about the origin. ModelPoly is rotated, translated and scaled by 
the matrices R, T and S to form a winPoly. It is winPoly as projected on the 
screen of the display 24 that the user sees. Initially, the matrices R, T, and S do 
not specify any rotation, translation or scaling of winPoly as compared to 
modelPoly. Then the user gradually adds information to each of R, T and S by 
performing sequences of geometrical transformations on winPoly. Say, for 
example, that the user rotates winPoly, thereby changing the visible face of 
winPoly (by convention, visible faces are defined by a line of sight that is parallel 
to the z coordinate axis and is negative). This rotation information is recorded 
in R, by multiplying R 0 by a matrix representing the rotation of winPoly (R, in this 
case - R n is the rotation matrix after n distinct rotations have been applied to 
modelPoly) Subsequently, winPoly is rotated again; this information is recorded 
in R 2 by multiplying R, by a matrix representing this second rotation of winPoly 
(the matrix generated by R," 1 * R 2 in this case). 

[0028] Some operations such as slicing and pivoting are recorded on 
modelPoly itself and not in the matrices that transform modelPoly to winPoly. 
These operations determine the clip state of modelPoly; that is, these operations 
determine all of current surfaces of modelPoly. Initially, this clip state includes 
the six faces and eight vertices of the parallelepiped. A new surface can then be 
added by slicing in from one of the original six planes along the normal of such 
original face to generate a new surface parallel to such original face. Then the 
original face can be discarded by cutting in from the first-added surface. A new 
surface can also be generated by pivoting about an arbitrary axis on one orignial 
face. After this pivoting operation, the clip state would include the six initial faces 
and the new added surface. 



[0029] After modelPoly is rotated, translated and scaled by the matrices 
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R, T and S to form winPoly, the coordinates of the visible faces of winPoly are 
projected to a two-dimensional image with lines connecting adjacent coordinates. 
The two-dimensional projection of winPoly is then sent to a raster submodute 38 
(Fig. 1 ). The raster submodule 38 maps image data onto the two-dimensional 
image of each visible faces of winPoly, by, for each ordered image property of 
the image data array that has associated coordinates on a visible surface, 
mapping such ordered image property onto such associated coordinates. First, 
the projected face is broken up into contiguous, 2 dimensional triangles. Next, 
each triangle is filled by first identifying the minimum y value and the maximum 
y value that lie in the triangle. Then, for these two points, and for each y value 
falling between these two points, a line segment is determined. One end of this 
line segment is a point having the smallest x integer value falling inside the 
triangle, while the other end is a point having the largest x integer value falling 
inside the triangle. The ordered image properties having associated spatial 
coordinates corresponding to this line segment are then mapped onto the line 
^ segment by the raster submodule. 

ill [0030] The projected two dimensional image of winPoly is projected onto 

J the screen of the display 24 where it can be seen by the user. Using the input 

device 30, the user can send commands to a user interface submodule 40, 
where these commands are interpreted and transmitted to the polytope module 
34. These commands include translate, rotate or scale, winPoly, in which case 
the polytope submodule 34 updates the matrices T, R and S as required. 
Alternatively, these commands include slice or pivot surfaces of modelPoly, in 
order to add new surfaces to modelPoly. These new surfaces of modelPoly will 
also appear in winPoly, after being rotated, translated and scaled by R, T and S, 
and from winPoly will be projected onto a two dimensional surface to be 
displayed on the display 24. 

[0031] The above-described geometrical manipulation as well as surface 

generation using slicing and pivoting are well known to those skilled in the art of 
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medical imaging and are widely used. Accordingly, no further detail regarding 
these imaging techniques will be provided herein. Using the above-described 
computer system, users can generate a series of sectional views, which may be 
parallel to one of the originally six faces or may be oblique sectional views. 
These sectional views can then be reviewed by a physician or other suitably 
skilled person to identify points of interest. Those sectional views that include 
points of interest will typically be retained by the physician, who will use them to 
help make a diagnosis. 

[0032] When reviewing image data using the computer 22, a user interacts 

with winPoly as displayed on the display 24, by translating, rotating and scaling 
winPoly, and by slicing or pivoting surfaces of modelPoly, which sliced and 
pivoted surfaces are automatically mapped onto winPoly. 

[0033] In one embodiment, as shown in Figure 2, the display 24 shows a 

graphical interface 40 which includes a volume view 42 and three individual 
sectional views 44, 46 and 48. The volume view 42 shown in Figure 2 contains 
a rectangular volume, the three visible sides of which comprise individual visible 
faces of winPoly displayed in perspective, as created by the raster submodule 38 
(Figure 1). Each visible face of the rectangular volume also shows a set of 
orthogonal cross-hairs. Each cross-hair represents a sectional plane 45, 47 and 
49 through the three dimensional image, the three orthogonal sectional planes 
intersecting within the three dimensional image. Each sectional view 44, 46 and 
48 is a display of a respective sectional plane 45, 47 and 49 corresponding to the 
cross-hairs shown in the volume view 42. In particular, using an x-y-z orthogonal 
coordinate system 41 , wherein the y-axis extends towards the top of the volume 
view 42, the x-axis extends towards the right side of the volume view 42, and the 
z-axis extends to the upper left of the volume view 42 (notionally, into the display 
24), it will be understood that the sectional view 46 corresponds to a sectional in 
a constant-z plane 47. Similarly, the sectional view 44 corresponds to a sectional 
in a constant-y plane 45 and the sectional view 48 corresponds to a sectional in 
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a constant-x plane 49. In Figure 3, the graphical interface 40 has a volume view 
42 shown in rendered format. 

[0034] Referring again to Figure 2, the cross-hairs shown on the visible 

faces of the rectangular volume in the volume view 42 may be manipulated by 
the user using the input device 30 (Figure 1 ) to adjust their position on the visible 
faces, and accordingly, adjust the position of the sectional plane through the 
three dimensional image. The sectional views 44, 46 and 48 show slices of the 
three dimensional volume at the user's selected location. In this manner the user 
may manipulate the cross-hairs to locate the sectional views 44. 46 and 48 that 
provide the clearest views of a feature of interest. 

[0035] A three-dimensional rendering of an image, such as that shown in 
the rendered volume view 42 in Figure 3, is typically partially obfuscated by 
unwanted artifacts. In the case of medical imaging, these unwanted artifacts 
may include tissues or features unrelated to the feature of interest. The example 
image shown in Figure 3 is from a fetal ultrasound scan. The volume view 42 
shows a view of the face 50 and part of the torso 52 of a fetus that is partially 
obscured by the uterus lining 54. The position of the uterus lining 54 above the 
face 50 of the fetus is seen in the sectional views 46 and 48. 

[0036] As will now be described in more detail, the present invention 

comprises a system and method for recognizing and selecting a region of interest 
in an image. In one embodiment, the system and method employs a volume 
view and selectable sectional views to assist the user to recognize a region of 
interest and to identify unwanted artifacts. The system and method also employs 
user-controlled region growing to select the region of interest and exclude the 
unwanted artifacts. 

[0037] Referring again to Figure 1, the computer system 22 includes a 

region growing submodule 36. The region growing submodule 36 is responsive 
to the input device 30 and maintains a growth matrix G containing the selected 
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region. The growth matrix G is used to update a scalpelling matrix C, which the 
polytope submodule 34 uses as a mask for modelPoly to select the region of 
interest for display. The region growing submodule 36 implements a region 
growing routine that grows a three dimensional region in the image starting from 
a seed point. 

[0038] Once the user has identified a desirable feature of interest, such as 

the face of a fetus, and has located sectional views that provide a reasonably 
clear depiction of the feature of interest showing some separation between it and 
any unwanted artifacts, the user will then wish to isolate a region containing the 
feature of interest and scalpel away the remainder of the image, in accordance 
with a method of the present invention, the user selects a point within the feature 
of interest on one of the sectional views using the input device 30. The system 
22 determines the spatial coordinates in modelPoly corresponding to the display 
coordinates selected by the user. The selected point is a seed point for region 
growing. 

[0039] Techniques for region growing for the selection of an area of 

interest within an image are known in the art of image processing. Simple region 
growing begins with a seed point in an image. From the seed point, the region 
growing routine analyzes each pixel surrounding the seed point and determines 
whether each pixel is a member of the region. The object of region growing is to 
select a region within an image based upon homogeneity criteria; thus, the region 
growing routine determines whether each surrounding pixel is a part of the region 
based the homogeneity criteria. The homogeneity criteria may specify that a 
pixel belongs to the region if the gray value of the pixel falls within a range of gray 
values which is based upon the gray value of the seed point. The range used 
depends upon the specific type of image being analyzed. 

[0040] After a first iteration, the region growing routine will have 

determined whether or not the pixels immediately surrounding the seed point are 
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members of the region or not. The next iteration and each subsequent iteration 
performs the same analysis for the pixels immediately adjacent the region 
identified in the previous iteration. Accordingly, the region expands outwards 
with each iteration until the routine determines that none of the surrounding 
pixels are members of the region. At each pixel along the edge of the region 
where the adjacent non-member pixels do not satisfy the homogeneity criteria, 
the region growing routine may be said to have reached a threshold. 

[0041 ] Those skilled in the art will be familiar with alternative methods and 
homogeneity criteria for determining whether a pixel is a member of the region, 
and with merging regions and other region growing techniques. Similarly, those 
skilled in the art will appreciate that the use of region growing may be employed 
in two dimensional images, three dimensional images or higher dimensional 
images. In a three-dimensional implementation, a region growing routine may 
analyze the voxels surrounding the seed point in a manner similar to the analysis 
of the pixels surrounding the seed point in a two-dimensional image. 

[0042] In conventional image processing, region growing is performed 
automatically by the region growing routine until the region reaches a threshold 
in all directions. A disadvantage of this method of segmentation is that the region 
may grow into unwanted areas if the pixels or voxels between the areas and the 
region of interest do not present a threshold. Accordingly, reliance on a 
threshold (homogeneity criteria) to stop the region growing tends to result in 
selection of unwanted areas due to leakage of the region. 

[0043] According to this aspect, the present invention provides for direct 

user control over the extent of the region growing through the input device 30. 
By manipulating the input device 30, such as a mouse or trackball, the user can 
control the number of iterations that the region growing routine performs. The 
physical displacement of the mouse or track ball is identified by the user interface 
submodule 40 and is used to increase or decrease an iteration variable. The 
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region growing submodule 36 uses the iteration variable to control the number 
of iterations that the region growing routine performs. Accordingly, beginning 
from a physical starting point, the user initiates the region growing by moving the 
mouse or trackball. The movement is detected by the user interface submodule 
40, which causes a corresponding increase in the iteration variable. In response 
to the increased iteration variable, the region growing submodule 36 causes the 
region growing routine to perform a further iteration, thereby increasing the size 
of the selected region. By ceasing to move the mouse or trackball, the user may 
stop the region growing routine even before the selected region reaches a 
threshold at each boundary point. Moreover, by withdrawing of the mouse or 
trackball towards its starting point, the iteration variable is decreased, causing 
the selected region to be retracted to a smaller region. Advantageously, direct 
user control in conjunction with a display of the growing region allows the user 
to prevent leakage of the selected region into unwanted artifacts. 

[0044] In one embodiment, the initial position of the mouse or other input 

device is recorded by the system 22 when the seed point is selected. 
Subsequently, the user interface submodule 40 detects movement of the mouse 
and assesses the direction in which the mouse has been moved as compared 
to its initial position. The user interface submodule 40 provides data to the region 
growing submodule 36, the data including whether the mouse has been moved 
and in what direction. In one embodiment, movement of the mouse to the right 
or left is indicative of a user instruction to expand or contract the region, 
respectively. Accordingly, the detection of mouse movement to the right causes 
the region growing submodule 36 to expand of the region by one or more 
iterations of the region growing routine, and mouse movement to the left causes 
the region growing submodule 36 to retract the region by one or more iterations 
of the region growing routine. 

[0045] According to another aspect of the present invention, the region 

growing routine performs a set number of iterations in response to movement of 
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the mouse by the user. For example, upon detecting movement of the mouse 
to the right, the region growing submodule 36 expands the region by five 
iterations. Following the fifth iteration, the region growing submodule 36 halts the 
region growing routine until further movement of the mouse is detected. Upon 
detecting further movement of the mouse to the right, the region growing 
submodule 36 causes the region growing routine to perform a further five 
iterations, and soon. Conversely, upon detecting movement of the mouse to the 
left after a set of iterations, the region growing submodule 36 causes the 
retraction of the region by five iterations. 

[0046] According to yet another aspect of the present invention, the region 
growing routine 36 includes a size limit which specifies a limit for the number of 
selected points (pixels or voxels). The region growing routine 36 continues to 
evaluate points and add selected points to the region until the selected number 
of points reaches the size limit, then the region growing routine 36 stops the 
growth of the region. The user controls the size limit through movement of the 
mouse, as described above. Specifically, the size limit in the case of a two- 
dimensional image is N 2 , and in the case of a three-dimensional image is N 3 , 
where N is an adjustable integer that may be increased or decreased in 
increments of 5. 

[0047] In the event that the region reaches a threshold at one or more 

boundary points, further movement of the mouse will not cause furtherexpansion 
of the region at those threshold boundary points. In the event that the region 
reaches a threshold in all directions, no further expansion of the region is 
possible. 

[0048] The growth matrix G and the scalpelling matrix C have a one-to-one 

correspondence with the modelPoly coordinate space and initially contain null 
values. Once the user selects a seed point and that seed point is translated to 
a point in the modelPoly coordinate space, a corresponding point in the growth 
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matrix G is set to 1 . As the region grows in the coordinate space in a pattern 
based upon the image property data contained in modelPoly, the corresponding 
points in the growth matrix G are set to 1 . The scalpelling matrix C is updated at 
each iteration by the growth matrix G, such that the scalpelling matrix C also 
contains a logical one at each coordinate system point identified as a member 
of the region and a logical zero at every other point. During the user-controlled 
region growing routine, the scalpelling matrix C is used as a mask to black out 
the points in modelPoly selected as members of the region. The user may 
monitor the growth of the region by observing the growth of the blacked-out 
region on the display 24. Following the termination of the region growing routine, 
the scalpelling matrix C is used as a mask to select only the points in modelPoly 
that are members of the region. The selected region of the image is then 
displayed to the user. 

[0049] The steps of the method and the operation of the system are 
described in more detail with reference to an example three dimensional image 
of a fetal ultrasound. 

[0050] Reference is again made to Figure 2, which shows a screen shot 

of the graphical interface 40 upon which is displayed the volume view 42 of the 
fetal image and the three orthogonal sectional views 44, 46 and 48. In operation, 
the user adjusts the crosshairs shown in the volume view 42 until the user 
recognizes a feature of interest and the feature of interest is clearly visible in one 
or more of the sectional views 44, 46 and 48. Optionally, the user may choose 
to have the volume view 42 rendered, as shown in Figure 3, to assist in 
identifying the feature of interest and in identifying unwanted artifacts. In the 
example screen shot shown in Figure 3, it will be appreciated that the feature of 
interest is the face 50 of the fetus and that the unwanted artifacts include the 
uterus lining 54 and other tissues located above the face 50 and to the left of the 
face 50. The x-y and y-z sectional views 46 and 48 provide a clear view of the 
fetus' face 50 separate from the unwanted artifacts 54. Accordingly, these views 
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enable the user to select a seed point well within the feature of interest and to 
monitor the growth of the region. To select a seed point, the user positions a 
mouse pointer or other input device indicator within one of the sectional views 44, 
46 and 48 at the desired point and selects it using the mouse button. 

[0051] Reference is next made to Figure 4, which shows a screen shot of 

the graphical interface 40 including a mouse pointer 56 located near the center 
of the x-y sectional view 46. The user has selected a seed point at the position 
of the mouse pointer 56 shown in Figure 4. A small dark region 58 is visible 
about the seed point, indicating that the mouse (not shown) has been moved a 
small distance causing a small growth in the region 58. Each dark pixel in the 
small dark region 58 is a pixel that has been blacked out because the 
corresponding point in the coordinate space has been identified as a member of 
the selected region. 

[0052] Reference is next made to Figure 5, which shows a screen shot of 

the graphical interface 40 after further growth of the selected region. The dark 
area 58 indicating points selected as members of the region has grown larger in 
response to further physical movement of the mouse (not shown) by the user. 
The dark area is now visible in each of the three sectional views 44, 46 and 48, 
allowing the user to monitor the growth of the selected region in each plane. 

[0053] In Figure 6, the dark area 58 has expanded beyond the fetus' face 

50 indicating that the region has leaked into unwanted artifacts, as shown in each 
of the sectional views 44, 46 and 48. Note the asymmetric growth of the dark 
area 58 due to the selected region meeting a threshold at various points in the 
three dimensional image space. Note also, in the volume view 42, that the dark 
area 58 appears in the visible plane, indicating that the region has reached the 
edge of the image space. 

[0054] The user recognizes that the region growing routine has performed 

too many iterations thereby causing growth of the selected region beyond the 
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feature of interest, namely the face 50 of the fetus. Accordingly, the user 
physically retracts the mouse, causing a decrease in the iteration variable and a 
corresponding retraction in the region. By physically moving the mouse, the user 
may adjust the iteration variable, thereby extending or retracting the region, until 
an optimal region is reached. The optimal region selects as much of the feature 
of interest as is possible without selecting a significant number of points 
corresponding to unwanted artifacts. Figure 7 shows a screen shot of the 
graphical interface 40 with the dark area 58 approximately covering the feature 
of interest in each of the sectional views 44, 46 and 48 without selecting a 
significant portion of the unwanted artifacts. 

[0055] Once the user is satisfied that the region growing has resulted in 

an optimal region, the system 22 will perform a scalpelling operation to remove 
all unselected areas of the image space. This operation is performed by applying 
the scalpelling matrix C as a mask to modelPoly. The scalpelling matrix C is 
updated from the growth matrix G so that the scalpelling matrix C selects only the 
coordinate points identified as members of the region. The resulting winPoly will 
contain only image data for the selected region. Figure 8 shows a screen shot 
of the graphical interface 40 following the scalpelling operation. As shown in the 
sectional views 44, 46 and 48, the scalpelling has removed all image data in the 
unselected area 62 surrounding the selected region of interest 60. The visible 
faces of the rectangular volume in the volume view 42 show no image because 
the region selected did not capture any coordinate points within the visible 
planes, i.e. the outer edges of the image space. 

[0056] The volume view 42 may then be rendered to show the rendered 

image 64 of the selected region of interest 60, as depicted in the screen shot 
shown in Figure 9. The user may then use all the existing features of the system 
22 to rotate, scale or translate the rendered image 64 of the fetus' face, as 
shown in Figure 10. A printing device may be attached to the computer system 
to print the rendered volume view 42. 
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[0057] In a further embodiment, the selected region is rendered in the 

volume view 42 at each iteration, allowing the user to monitor the growth of the 
region in that view as well. Reference is now made to Figures 11 through 17 
which show a screen shot of the graphical interface 40 illustrating an 
embodiment of the invention wherein the selected region is rendered in the 
volume view 42 during the region growing process. 

[0058] Figure 1 1 shows a screen shot of the graphical interface 40 upon 

which is displayed the volume view 42 of the fetal image and the three orthogonal 
sectional views 44, 46 and 48. The fetus' face 50 and the uterus lining 54 are 
visible in the rendered volume view 42, wherein it will be appreciated that the 
uterus lining 54 partially obscures the fetus' face 50. 

[0059] Once again, the feature of interest that the user wishes to view is 

the fetus' face 50. Accordingly, the user positions the mouse pointer 56 within 
the area of the feature of interest in one of the sectional views 46, as shown in 
Figure 1 2. The user then selects a seed point within the feature of interest using 
the mouse pointer 56. The selection of a seed point begins the region growing 
process. To begin with, the region consists solely of the seed point and thus the 
rendered image of the selected region in the volume view 42 begins as a single 
point. 

[0060] Reference is now made to Figure 1 3 which shows a screen shot of 

the graphical interface 40 after the user has moved the mouse (not shown) a 
small distance. The mouse pointer 56 and a small dark area 58 representing the 
selected points of the region are visible in the x-y sectional view 46. A rendered 
view 66 of the selected region is shown in the volume view 42. 

[0061] With further movement of the mouse, the user causes the region 
growing routine to expand the selected region, and thus the dark area 58, as 
shown in Figure 14. As described above, the growing dark area 58 
corresponding to points of the selected region is visible and may be monitored 
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by the user in the three sectional views 44, 46 and 48. In this embodiment, the 
growth of the region may also be monitored by the user through the rendered 
view 66 of the region shown in the volume view 42. 

[0062] Reference is now made to Figure 15, which shows a screen shot 

of the graphical interface 40 after the user permitted the growth of the region to 
extend beyond the desired region of interest. As shown in the sectional views 
44, 46 and 48, the dark area 58 has extended into unwanted artifacts and 
features. This is also apparent to the user from the volume view 42 which shows 
a rendered view of the fetus' face 50 and the uterus lining 54. 

[0063] Accordingly, the user manipulates the mouse to cause the region 

to retract, as shown in Figure 16. The volume view 42 showing the rendered 
view 66 of the selected region contains the fetus' face and some small amount 
of excess material. Once the user is satisfied with the selected region, the 
unselected points are scalpelled away and the remaining points are displayed, 
as shown in Figure 17. In the volume view 42, the user has a clear and 
unobstructed rendered view of the fetus' face 50. 

[0064] Although the embodiments of the subject invention described 
above involve the selection of a feature of interest using user-controlled region 
growing, and following which unselected areas are scalpelled away, those skilled 
in the art will understand that the subject invention may be implemented such 
that the selected region is scalpelled away and the unselected area is retained. 
In this alternative embodiment, the user selects an undesired feature of interest 
rather than a desired feature of interest. A user-controlled region is then grown 
within the undesired feature, following which the region is scalpelled away leaving 
the remainder of the image data, including the desired feature. Those skilled in 
the art will also understand that multiple seed points may be selected such that 
multiple regions are grown, either in succession or in parallel. 

[0065] In addition to the particular embodiments described above, those 
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skilled in the art will appreciate that the present invention may be implemented 
using various region growing techniques and various techniques for relating the 
manipulation of the input device to the extent of the region growth. 

[0066] While the subject invention has been described such that the seed 

point is selected from a sectional view of a three dimensional image, it will be 
appreciated by those skilled in the art that alternative methods may be employed. 
For example, the seed point may be selected from the volume view. 

[0067] It will also be appreciated that although the embodiment of the 
present invention described uses a Cartesian coordinate system for defining and 
manipulating the three dimensional images, other coordinate systems could be 
used. For example, one could use a circular cylindrical coordinate system or a 
spherical coordinate system. 

[0068] Using the foregoing specification, the invention may be 

implemented as a machine, process or article of manufacture by using standard 
programming and/or engineering techniques to produce programming software, 
firmware, hardware or any combination thereof. 

[0069] Any resulting program(s), having computer readable program code, 

may be embodied within one or more computer usable media such as memory 
devices, transmitting devices or electrical or optical signals, thereby making a 
computer program product or article of manufacture according to the invention. 
The terms "article of manufacture" and "computer program product" as used 
herein are intended to encompass a computer program existent (permanently, 
temporarily or transitorily) on any computer usable medium. 

[0070] A machine embodying the invention may involve one or more 

processing systems including, but not limited to, central processing unit(s), 
memory/storage devices, communication links, communication/transmitting 
devices, servers, I/O devices, or any subcomponents or individual parts of one 
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or more processing systems, including software, firmware, hardware or any 
combination or sub-combination thereof, which embody the invention as set forth 
in the claims. 

[0071] One skilled the art of computer science will be able to combine the 

software created as described with appropriate general purpose or special 
purpose computer hardware to create a computer system and/or computer sub- 
components embodying the invention and to create a computer system and/or 
computer sub-components for carrying out the method of the invention. 

[0072] Accordingly, it will be apparent that the present invention may be 

implemented in other specific forms without departing from the spirit or the 
essential characteristics thereof. In particular, while the invention has been 
described in the context of medical imaging generally, and ultrasound imaging 
in particular, it will be apparent to those skilled in the art that the invention is 
applicable in other imaging contexts. Additionally, while the invention has been 
described in connection with a user input device such as a manually operated 
computer mouse, it will be apparent to those skilled in the art that the invention 
may be implemented in connection with any other input device. The presently 
discussed embodiments are considered to be illustrative and not restrictive, the 
scope of the invention being indicated by the appended claims rather than the 
foregoing description, and all changes that come within the meaning and range 
of the claims are therefore intended to be embraced. 



